Describing the state of a research network: A mixed methods approach to network evaluation.

Diabetes Action Canada Strategy for Patient-Oriented Research (SPOR) Network in Chronic Disease was formed in 2016 and is funded primarily through the Canadian Institutes of Health Research (CIHR). We propose a novel mixed-methods approach to a network evaluation integrating the State of Network Evaluation framework and the Canadian Academy of Health Sciences (CAHS) preferred framework and indicators. We measure key network themes of connectivity, health and results, and impact and return on investment associated with health research networks. Our methods consist of a longitudinal cross-sectional network survey of members and social network analysis to examine Network Connectivity and assess the frequency of interactions, the topics discussed during them, and how networking effectively facilitates interactions and collaboration among members. Network Health will be evaluated through semistructured interviews, a membership survey inquiring about satisfaction and experience with the Network, and a review of documentary sources related to funding and infrastructure to evaluate Network Sustainability. Finally, we will examine Network Results and Impact using the CAHS preferred framework and indicators to measure returns on investment in health research across the five domains of the CAHS framework, which include: advancing knowledge, capacity building, informing decision making, health impact, and economic and social impact. Indicators will be assessed with various methods, including bibliometric analyses, review of relevant documentary sources (annual reports), member activities informing health and research policy, and Patient Partner involvement. The Network Evaluation will provide members and stakeholders with information for planning, improvements, and funding future Network endeavors.

Inter-individual relationships within a Canadian SPOR research network: a social network study.

BACKGROUND: Efforts have been made by health research granting agencies to bring research closer to patients' concerns. In Canada, such efforts were formalized in 2011 with the funding of the Strategy for Patient-Oriented Research (SPOR)'s research networks to address research priorities identified by patients and accelerate the translation of research findings into patient care and health care policy. Among these networks, SPOR Diabetes Action Canada (DAC) has created patient-partner circles to facilitate their integration within the network. The nature of the relationships within this atypical patient-oriented research network is systematically explored in this paper. METHODS: A cross-sectional social network study was conducted among the SPOR DAC's network members to examine inter-individual interactions, and the topics discussed the most between members. Descriptive data analyses were conducted to explore which discussion topics were discussed most among members whose primary roles were research, administration, governance, and patient representation. RESULTS: The response rate was 51.9%, providing data on 76.5% of the maximum number of connections in the network. The survey captured 2763 inter-individual relationships. Responses to a sub-question inserted in the survey show that 482 of these relationships (17,4%) existed before joining the network in collaboration on a research project. Most ties captured in the survey were yearly or quarterly, while few relationships were monthly, weekly, or daily. In measured relationships, members discussed several topics, the most frequent being scientific research, patient engagement, network coordination and governance, and operations and management. The topics associated with the most significant proportion of relationships captured in the survey were scientific research (45.4%) and patient engagement (40.7%). Management & operations and governance & coordination follow, corresponding to 24.3 and 23.9% of the captured relationships. All discussion topic subnetworks were either somewhat or highly centralized, meaning that relationships were not equally distributed among members involved in these discussions. Of the 1256 relationships involving exchanges about scientific research, 647 (51.5%) involved a researcher, 419 (33.3%) an administrator, 182 (14.5%) a patient partner, and 82 (6.5%) a member whose primary role is network governance. CONCLUSIONS: Scientific research and patient engagement were the most common topics discussed, consistent with the patient-centered research at the heart of the SPOR Diabetes Action Canada network. The study identified several relationships where a patient partner has discussed scientific research with a researcher. However, relationships involving research discussions were three times more common between a researcher and an administrator than between a researcher and a patient partner, although twice as many patient partners as administrators participated in the survey. The institutionalization of patient-partner involvement in large research networks is an evolving practice for which optimal engagement methods are still being explored.

The Impact of Information Presentation and Cognitive Dissonance on Processing Systematic Review Summaries: A Randomized Controlled Trial on Bicycle Helmet Legislation.

Background: Summaries of systematic reviews are a reference method for the dissemination of research evidence on the effectiveness of public health interventions beyond the scientific community. Motivated reasoning and cognitive dissonance may interfere with readers' ability to process the information included in such summaries. Methods: We conducted a web experiment on a panel of university-educated North Americans (N = 259) using a systematic review of the effectiveness of bicycle helmet legislation as a test case. The outcome variables were the perceived tentativeness of review findings and attitude toward bicycle helmet legislation. We manipulated two types of uncertainty: (i) deficient uncertainty (inclusion vs. non-inclusion of information on limitations of the studies included in the review) and (ii) consensus uncertainty (consensual findings showing legislation effectiveness vs. no evidence of effectiveness). We also examined whether reported expertise in helmet legislation and the frequency of wearing a helmet while cycling interact with the experimental factors. Results: None of the experimental manipulations had a main effect on the perceived tentativeness. The presentation of consensual efficacy findings had a positive main effect on the attitude toward the legislation. Self-reported expertise had a significant main effect on the perceived tentativeness, and exposing participants with reported expertise to results showing a lack of evidence of efficacy increased their favorable attitude toward the legislation. Participants' helmet use was positively associated with their attitude toward the legislation (but not with perceived tentativeness). Helmet use did not interact with the experimental manipulations. Conclusions: Motivated reasoning and cognitive dissonance influence a reader's ability to process information contained in a systematic review summary.

A scoping review of knowledge syntheses in the field of evaluation across four decades of practice.

This scoping review of 62 knowledge syntheses published in evaluation-focused journals between 1979 and May 2018 provides a portrait of synthesis practices and their evolution in the mainstream of the field of evaluation. Concerns surrounding the production of knowledge syntheses to answer policy questions are not new in the field of evaluation. However, during this last decade, knowledge synthesis methods have expanded as a means to go beyond the limits and constraints of singular evaluations. This scoping review reveals and discusses two key issues with regards to the expansion of knowledge synthesis practices within the field of evaluation: the diversity-and muddling- of methodological practices and synthesis designs, and the frequent omission of quality appraisals.